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Executive Summary 



Building on research and practice, the National Center on Educational Outcomes (NCEO) has 
revisited and updated its 2001 document that identified principles and characteristics that un- 
derlie inclusive assessment and accountability systems. This report on a principled approach to 
accountability assessments for students with disabilities reflects what we have learned during 
the past seven years. 

The principles provide a vision for an inclusive system of assessments used for system account- 
ability. We address state and district K-12 academic content assessments designed for system 
accountability and focus specifically on all students with disabilities, including targeted groups 
of students within this group (e.g., English Language Learners with disabilities). Multiple stake- 
holders who share the common goal of improving educational outcomes for all students have 
reviewed and provided comments on the principles and characteristics presented here. 

This report presents six core principles, each with a brief rationale, and specific characteristics 
that reflect each principle. The principles are: 

Principle 1. All students are included in ways that hold schools accountable for their 
learning. 

Principle 2. Assessments allow all students to show their knowledge and skills on the same 
challenging content. 

Principle 3. High quality decision making determines how students participate. 

Principle 4. Public reporting includes the assessment results of all students. 

Principle 5. Accountability determinations are affected in the same way by all students. 

Principle 6. Continuous improvement, monitoring, and training ensure the quality of the 
overall system. 

Several technical assistance tools to support the principles are in development. These include 
state self-evaluation tools, references for key topic areas, and one -page summaries on each topic 
covered by the principles. 
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Foreword 



The National Center on Educational Outcomes (NCEO) periodically examines the assessment 
and accountability context in the U.S. to determine how students with disabilities are included 
in state implementation. We specifically look at underlying policy assumptions and actions, 
as well as potential consequences for students with disabilities. We have done this for two de- 
cades, and have documentation of our work in reports that are available on the NCEO Web site 
(www.nceo.info). We believe that it is possible to enhance the positive consequences of assess- 
ments used for system accountability with students with disabilities and reduce their negative 
consequences through systematic attention to assumptions in the design, implementation, and 
continuous improvement of assessments and related accountability systems. 

This publication is primarily for audiences in state departments of education, especially for the 
leadership in assessment and special education offices and their partners who work on large- 
scale assessments for the purpose of system accountability. This publication is also targeted 
toward measurement experts who may sit on state technical advisory committees, and the testing 
contractors who develop large-scale assessments for system accountability. The information in 
this document is relevant to many others as well, including policymakers, administrators, and 
parents. 

We believe that the NCEO principles and their accompanying characteristics are generally 
consistent with the Standards for Educational and Psychological Testing produced by the 
American Educational Research Association, American Psychological Association, and National 
Council on Measurement in Education. Although the NCEO principles do not cover the breadth 
of issues that the Standards cover, in many ways, they go beyond the Standards because they 
address more comprehensively the inclusion of students with disabilities. We believe that the 
NCEO principles also are consistent with the Accessibility Principles for Reading Assessments 
document developed by the National Accessible Reading Assessment Projects, which focuses 
exclusively on reading. 

We last published a set of principles outlining foundational assumptions that support positive 
consequences of inclusion in assessments in 2001. We have learned much since that time. We 
examined the research and practice of the past seven years to develop the principles and char- 
acteristics in this document. Core resources (i.e., foundational articles from the wider field of 
large-scale assessment, comprehensive literature reviews, or key policy analyses) from this 
research and practice base are included at the end of this document. 

Technical assistance tools to support use of the principles in state level self-evaluation and im- 
provement efforts will be developed. In addition, a comprehensive reference list by key topic 
areas (universal design of assessments, accommodations, alternate assessment, reporting, and 
accountability) will be provided. We also will provide one-page summaries of what we know 
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about each of these topics so that the principles are more accessible to multiple audiences. 
The summaries will target audiences beyond state offices and be suitable for local education 
agency staff, parents, post- secondary partners, and the general public. 
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Overview 



There has been a shift in thinking about schools and schooling in the United States — a shift 
toward high standards for learning for all children, including children with disabilities. National 
and state policies have made use of the tools of standards-based assessments and accountability 
systems to push practices for improved achievement, including for students with disabilities, 
access to the general curriculum based on the same goals and standards as for all other students, 
and accountability approaches that promote positive results. We have realized that an inclusive 
system of assessments used for system accountability is one that neither obscures nor discounts 
what all students really do know and are able to do. 

The purpose of this document is to provide a vision for such a system. It is a document that states 
and districts can use in developing and producing a new assessment. It is also a document that 
states and districts can use in examining their own assessment for use in system accountability, 
and in thinking about revisions to their system. We use the term “principled” intentionally in 
the title to communicate that what we are presenting is a view of how things should be in an 
inclusive system. We provide a set of characteristics for each principle; these characteristics 
further define what it takes to realize a principle in practice. 

The principles address state and district K-12 academic content assessments designed for 
system accountability. Most commonly, these are assessments of reading/English language 
arts, mathematics, science, social studies, and any other academic content that states or districts 
might assess on a large-scale basis. 

The principles specifically address the needs of students with disabilities. Given the increas- 
ing linguistic and cultural diversity of the K-12 student population nationwide, any attempt to 
address the needs of students with disabilities must also specifically address the needs of lin- 
guistically and culturally diverse students with disabilities. English Language Learners (ELLs) 
with disabilities, in particular, are important to include here because they require additional 
considerations for their inclusion in assessments due to their limited English skills. Although 
we developed the principles to apply to ELLs with disabilities as well as students with disabili- 
ties who are native English speakers, we believe that they apply to many other populations of 
students as well. Nevertheless, the research basis that we cite, our narrative, and the technical 
assistance tools reflect our focus on students with disabilities. 

Underlying the vision in this document is the belief that students with disabilities can and should 
be expected to achieve the same academic outcomes as their peers without disabilities. In order 
for that to occur, high quality instruction, access to the general curriculum based on the same 
curriculum standards as used for typical students, and systematic standards-based formative and 
summative assessments must be in place to allow these students to achieve in spite of barriers 
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related to disabilities. Our schools must be structured to allow students with disabilities to avoid 
the barriers that their disabilities create when accessing the curriculum and when demonstrating 
what they know and can do on assessments. Success in doing so is a critical step on the path 
toward life-long success. 

We believe that large-scale assessments can be designed, implemented, and improved over 
time to ensure high quality measurement of all students’ achievement of state-identified content 
standards for the purpose of system accountability. Our principles and characteristics describe 
quality indicators of such assessments. For students to actually achieve at high levels on these 
assessments, high quality professional development for all teachers and standards-based in- 
struction for all students must be in place. Although high quality professional development 
and instruction are essential but sometimes neglected components of standards-based reform 
designed to ensure success for all students, we are focused here on the assessment and account- 
ability systems surrounding standards-based systems. 

The principles do not address formative assessments, assessments used for progress monitor- 
ing, or benchmark testing, unless those assessments have been designed to be used for system 
accountability, or are being used for system accountability. Similarly, the principles do not 
specifically address graduation examinations or other promotion examinations, in part because 
these types of assessment are designed for individual student accountability and require ad- 
ditional considerations. The focus of the principles here is on those assessments used for 
system accountability (school, district, state), and specifically focuses on all students with 
disabilities and targeted groups of students within that population. 



Six Core Principles 

Each principle presented here reflects an essential quality of a good inclusive assessment sys- 
tem. The six principles were developed to reflect best practice as suggested by policy work 
and research over the past decades. They are not simply a check on compliance with legal 
requirements, although they generally are consistent with the requirements of current federal 
laws governing special education and Title I services (i.e., IDEA 2004; NCLB 2001). For each 
principle, specific characteristics are presented — these provide more precise information about 
specific elements of the quality identified by the principle. Each of the characteristics also is 
supported by a rationale statement. 

The graphic summary of the six core principles (see Figure 1) is meant to be used in conjunction 
with the more detailed explanations in the text. The graphic shows that Principle 1 is a core goal 
that is supported by the other five principles, with Principle 6 noting that all the principles are 
placed in a dynamic system that needs continuous oversight and improvement. 
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The six principles for including students with disabilities in large-scale assessments for system 
accountability are: 

Principle 1. All students are included in ways that hold schools accountable for their 
learning. 

Principle 2. Assessments allow all students to show their knowledge and skills on the same 
challenging content. 

Principle 3. High quality decision making determines how students participate. 

Principle 4. Public reporting includes the assessment results of all students. 

Principle 5. Accountability determinations are affected in the same way by all students. 

Principle 6. Continuous improvement, monitoring, and training ensure the quality of the overall 
system. 



Figure 1. Principles for Including Students with Disabilities in Large Scale Assessments for 
System Accountability 
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Principle 1 . All students are included in ways that hold schools accountable for their 
learning. 

All students are included in large-scale assessments for system accountability in ways that yield 
defensible inferences about student learning. There is no aspect of the assessment system or 
the accountability system that excludes a group of students, such as students with disabilities, 
students who are English language learners, ELLs with disabilities or other students who are 
highly mobile, disadvantaged, or of minority status. Every student is represented in the assess- 
ment system, the reporting system, the accountability system, and school improvement efforts. 
This principle reflects a belief that all students can be successful when there is systems level 
commitment to build the capacity for student success in every school and classroom. Including 
all students in assessment and accountability systems will not ensure their success on its own, 
but their inclusion can ensure that achievement data include all students before the data are used 
to target school improvement resources. 

When fully inclusive assessment systems are developed for system accountability, care is taken 
to ensure that all students can, in fact, demonstrate their skills and knowledge. This means that 
policies and practices result in system components that support more accurate inferences for 
students who have disabilities of all types. Decisions about each system component reflect a 
deep understanding of varied student characteristics that affect learning as students move toward 
proficiency in the grade-level content. The validity of the system is assured through assessment 
and accountability options that specifically address the implications of these varied student 
learning characteristics. Three characteristics support Principle 1 (see Table 1). 

Table 1. Principle 1 and Its Characteristics 



Principle 1. All students are included in ways that hold schools accountable for their 
learning. 

Characteristic 1.1. All students are included in every aspect of assessment for system 
accountability. 

Characteristic 1.2. Stakeholders with expertise and experience in varied student learning 
characteristics collaborate on all aspects of the assessment system to ensure that all students can 
show what they know and can do. 

Characteristic 1.3. The validity of the system is assured through technically defensible 
assessments that address the implications of varied student learning characteristics. 
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Characteristic 1.1. All students are included in every aspect of assessment for system account- 
ability. 

Every aspect of assessment for accountability includes not only full participation of all students in 
the assessments, but also in the reporting of data, determination of accountability measures, and 
use of data for school improvement. This characteristic reinforces the need to provide concrete 
methods of linking performance data reports for all students to the school improvement process, 
as well as to the accountability processes defined at state and district levels. The state and district 
provide flexible tools to allow school improvement teams to disaggregate performance data and 
to answer specific questions about the performance of a subset of students. Special attention is 
given to ways to disaggregate data within the group of students with disabilities, including ag- 
gregating by category of disability and for students with multiple identifiers, such as for ELLs 
with disabilities. Being able to further break down data by within category of disability (e.g., 
for students with learning disabilities, specifying nature of the barrier) or by language group 
for ELLs with disabilities, further supports use of data for improvement of assessments and 
instruction. 

State and district supports to schools considered in need of improvement include specific strate- 
gies designed to increase the performance of students who have not as yet achieved proficiency 
in the grade-level content. Inclusive systems assure that all students are included in the benefits 
of such supports. 



Characteristic 1.2. Stakeholders with expertise and experience in varied student learning 
characteristics collaborate on all aspects of the assessment system to ensure that all students 
can show what they know and can do. 

Inclusive practices start before the development of assessment systems, and continue through- 
out use of the assessment and accountability systems, with insights and oversight from key 
stakeholders who have skills and knowledge of academic standards, instructional processes, 
and understanding about the varied learning needs of all students. Key stakeholders include 
partners from general education, special education, English as a second language or bilingual 
education programs, curriculum, assessment and administrative personnel, parents, advocacy 
groups, related service providers (e.g., speech/language pathologists), and community members 
as appropriate. Educational professionals across disciplines and stakeholders representing var- 
ied student subgroups are essential partners in shaping the development of assessment systems 
that appropriately address varied learner characteristics in the context of a standards-based 
approach. They also must be active partners in monitoring the consequences of the assessment 
systems to ensure continuous improvement. This process will result in assessments that yield 
more defensible inferences about all students. 
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This collaboration yields improved student outcomes when these partners contribute to all 
aspects of standards-based systems, not just large-scale assessments for system accountability 
purposes. Their role includes advising on development and revision of content and achievement 
standards and systematic alignment of curriculum and instruction to the standards, including 
ensuring that formative assessments also allow all students to show what they know. These 
partners can help ensure coherent and aligned standards-based systems that result in improve- 
ment of outcomes for all students. 



Characteristic 1.3. The validity of the system is assured through technically defensible assess- 
ments that address the implications of varied student learning characteristics. 

This characteristic ensures that assessment design processes build on understanding how all 
students learn and show what they know. It requires careful consideration of varied student 
learning characteristics in the design of assessment options that yield defensible inferences about 
the learning of all students regardless of their unique needs. This view may require rethinking 
overall assessment design for fully accessible assessments and the development of improved 
accommodation policies and alternate assessment options. 

When innovative methods of assessment for unique learners are considered, care is taken in the 
application of traditional measurement conventions. When traditional measurement conventions 
do not match the assessment well, analogous and rigorous technical strategies are implemented 
to ensure the validity of the assessment. 



Principle 2. Assessments allow all students to show their knowledge and 
skills on the same challenging content. 

Assessment systems are designed and developed in ways that allow all students to demonstrate 
their knowledge and skills on the content and achievement standards for their enrolled grade. 
This principle indicates that all students with disabilities participate in an assessment system 
that is appropriately designed and developed to measure enrolled grade-level content, regardless 
of the nature or severity of their disability or whether they are learning English. Aspects of the 
system include best practices in the creation of accessible general assessments, including align- 
ment to state standards, flexible approaches to meeting intended constructs, universal design 
principles, appropriate use of accommodations, and alternate assessments. Four characteristics 
support Principle 2 (see Table 2). 
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Table 2. Principle 2 and Its Characteristics 



Principle 2. Assessments allow all students to show their knowledge and skills on the same 
challenging content. 

Characteristic 2.1. All students in all settings who receive special education services are included 
in their enrolled grade level assessment in some way (e.g., in general, accommodated, or alternate 
assessment), regardless of the nature of disability or special needs. 

Characteristic 2.2. All assessments are designed from the beginning with a focus on accessibility 
for all students. 

Characteristic 2.3. Accommodation policies are informed by the defined construct to be 
measured, available research findings, and the purpose of the assessment. 

Characteristic 2.4. Alternate assessments are used to assess the knowledge and skills of students 
whose disabilities are a barrier to demonstrating knowledge and skills in general assessments 
under standard conditions or with allowable accommodations. 



Characteristic 2.1. All students in all settings who receive special education services are in- 
cluded in their enrolled grade level assessment in some way (e.g., in general, accommodated, 
or alternate assessment), regardless of the nature of disability or special needs. 

The definition of “all” students includes all students who receive educational services in any 
setting. This includes students in traditional public school placements, and students who change 
schools or placements, as well as all students receiving federally funded educational services 
in non-traditional settings such as students in home schools, private schools, charter schools, 
state-operated programs, in the juvenile justice system, or any other setting where these educa- 
tional services are provided, with no exceptions because of the nature of disability or specialized 
services and supports required. 



Characteristic 2.2. All assessments are designed from the beginning with a focus on acces- 
sibility for all students. 

Creating an accessible assessment involves knowledge of needs of the full range of students to be 
tested along with careful scrutiny of intended constructs and design of assessments. Promising 
practice for accessible assessments includes reviewing assessments for alignment to standards 
and universal design elements, disaggregating assessment results at the whole-test and item 
level, and precisely defining constructs measured on assessments. Accessible assessments are 
reviewed for adherence to universal design elements, use data-based decision making for the 
inclusion of particular items (including statistical and qualitative studies on the impact of items 
on particular populations), and clearly describe what are the intended constructs of items as well 
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as “built in” accommodations students may use (such as defining which items allow calcula- 
tors for all students). Such transparency in desired student knowledge allows for clear policy 
and practice about what types of technology, human assistance, or other flexible approaches to 
assessment will and will not affect the validity of the assessment. 



Characteristic 2.3. Accommodation policies are informed by the construct to be measured, 
available research findings, and the purpose of the assessment. 

Policies that indicate which changes in testing materials or procedures can be used during as- 
sessments, under which conditions, and whether the use of the accommodations or modifica- 
tions might have implications for scoring or aggregation of scores are set by states. These may 
change by student characteristic. For example, ELLs with disabilities have access to allowable 
accommodations for both ELLs and students with disabilities. It is the responsibility of state 
leaders to gather stakeholders and technical advisors to review the purpose of the assessment 
and the constructs to be measured, along with available research findings to determine which 
accommodations allow for valid inferences. 



Characteristic 2.4. Alternate assessments are used to assess the knowledge and skills of students 
whose disabilities are a barrier to demonstrating knowledge and skills in general assessments 
under standard conditions or with allowable accommodations. 

There is a small number of students who require alternate assessments to the general assessment 
to demonstrate achievement. Data-based strategies are used to determine who the students are 
who cannot show what they know on the general assessment, why that is the case, and how 
their instructional opportunities influence assessment decisions. Before alternate assessments 
are created, states make decisions on how alternate assessment data will aggregate with gen- 
eral assessment data. The end goal of alternate assessments is to serve the need for appropriate 
measurement of particular students, so that all students are part of the state’s accountability 
system. Typically students who participate in alternate assessments are those whose disability 
precludes them from demonstrating knowledge in general assessments under standard conditions 
or with allowable accommodations. Alternate assessments are used for a very small segment of 
the population, and are properly designed and implemented. 



Principle 3. High quality decision making determines how students participate. 

Decisions about participation and accommodation of students in the assessment system are 
based on knowledge of student characteristics and needs, combined with knowledge of the goals 
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and purposes of accountability testing. This principle reflects the need for thoughtful decisions 
about how each student participates in the assessment system. An underlying assumption is 
the importance of high expectations while ensuring that each student can show what he or she 
knows and is able to do. Participation decisions are made by the IEP team with full knowledge 
of the implications of the decision. Established processes ensure that IEP teams have access 
to training and knowledge needed to make appropriate decisions for these students, regardless 
of the nature or severity of disability or whether they are learning English. Four characteristics 
support Principle 3 (see Table 3). 

Table 3. Principle 3 and Its Characteristics 



Principle 3. High quality decision making determines how students participate. 

Characteristic 3.1. Decisions about how students participate in the assessment system are based 
on how students show knowledge and skills within the context of accountability testing — not on the 
student’s instructional program, disability category, or current level of functioning. 

Characteristic 3.2. Assessment accommodations are available to all students, and decisions about 
accommodations are based on an individual student’s accommodation needs and experiences and 
what the assessment is designed to measure. 

Characteristic 3.3. Clear policies, guidelines, procedures, and training on assessment participation 
decision making are provided for all decision-making partners. 

Characteristic 3.4. The IEP team annually reviews and documents assessment participation and 
accommodation decisions on an individual student basis for each state and district assessment. 



Characteristic 3.1. Decisions about how students participate in the assessment system are based 
on how students show knowledge and skills within the context of accountability testing — not on 
the student’s instructional program, disability category, or current level of functioning. 

Historically, students with disabilities were excluded from assessments. As states and districts 
require that they participate in assessments, it may be tempting to try to protect students, keep 
them in easy levels of instruction and assessment, or let low expectations guide decisions. These 
temptations are avoided in an inclusive assessment system. Participation guidelines with decision- 
making criteria are developed to determine the ways in which individual students participate in 
the assessment system in order to show what they know. The needs of individual students and 
the purpose of the assessment are considered when decisions are made. 
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Characteristic 3.2. Assessment accommodations are available to all students, and decisions 
about accommodations are based on an individual student’s accommodation needs and experi- 
ences and what the assessment is designed to measure. 

All students have strengths and needs that result in the different ways they access instruction 
and assessment. Need is the major determinant of whether accommodations are used with any 
student (with or without identified disabilities), both for instruction and assessment. Reason- 
able decisions are made about certain accommodations that are used for instruction but are not 
appropriate for assessments because they confound the construct being measured. It is possible 
that some accommodations are appropriate for assessment but not for instruction. For example, 
a student may need training in how to use a type of technology that is not used for instruction 
but that enables the student to meaningfully access an assessment 



Characteristic 3.3. Clear policies, guidelines, procedures, and training on assessment partici- 
pation decision making are provided for all decision-making partners. 

State policies, guidelines, and procedures for assessment participation decision making are 
developed in collaboration with key stakeholders, are written in plain language that communi- 
cates clearly, and are provided to all partners in the decision-making process (IEP teams, 504 
teams, English as a Second Language or bilingual education planning partners, or any other 
stakeholders who contribute to these decisions for any student). There is clear articulation of 
specific issues that apply to classroom assessment and those that apply to large-scale assess- 
ment for accountability, with careful delineation of similarities and differences, and implications 
of specific decisions for the student and for the school. All of these policies, guidelines, and 
procedures reflect a commitment that choices being made for each student must promote both 
access to and high achievement in the student’s enrolled grade curriculum, a curriculum based 
on grade- specific content and achievement standards. 

These materials are supported by training designed to meet the needs of all partners, and em- 
phasize the linkage of curriculum, instruction, and assessment. Options for training are flexible 
and varied to allow all partners— parents, teachers, related service providers, and student as 
appropriate — to choose formats and schedules that meet the student’s needs. 



Characteristic 3.4. The IEP team annually makes, documents, and regularly reviews assess- 
ment participation and accommodation decisions on an individual student basis for each state 
and district assessment. 

Decisions about participation in one particular state or district assessment may be different from 
decisions about participation for another assessment that has a different purpose or different 
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format. The membership of every IEP team includes people who know the student and are in 
the best position to understand the issues that affect assessment for that student. An English 
as a Second language or bilingual education teacher plays a key role in making decisions for 
ELLs with disabilities. Parents and the student, when appropriate, are essential members of the 
team. Additionally, there may be other people not typically on the IEP team who have insight 
into the student’s needs; they should be consulted about decisions as well. These people may 
include the student (if the student is not already participating on the team), paraprofessional, 
counselor, psychologist, caretaker, and others. 

Participation decisions made by the IEP team for each state and district assessment, and the 
team’s rationale for the decisions, are made year by year, or more frequently if needed. IEP 
documentation of these decisions provides an important record of the individual student’s needs 
and strengths. These decisions are reviewed and changed as appropriate with the development 
of each annual IEP to reflect changing student needs and skills, and to reflect changes in the 
assessment system. Although IEPs often are developed all year on a schedule that may not co- 
incide with planning for state and district assessments, decisions are made at the IEP meeting 
that most closely precedes each assessment so that the most appropriate decisions are made. 



Principle 4. Public reporting includes the assessment results of all 
students. 

Public reporting is the first level of accountability for the results of students with disabilities. The 
philosophy underlying this principle is that every student counts and in a well-functioning system, 
the system itself is held accountable for every student. This philosophy is reflected through the 
inclusion of student results in public reports. Regardless of how students with disabilities are 
assessed — with or without accommodations, with a native language version, or in an alternate 
assessment — their results are reported. If their results are not reported due to technical adequacy 
issues (for example, the student used a testing modification), or due to issues with the testing 
company handling of tests, or due to student absence, or for some other reason, all students’ 
participation are still accounted for in the reporting system. A well- functioning system is flexible 
and allows for further disaggregation so that groups of students with multiple identifiers, such 
as ELLs with disabilities, are clearly reported. Although the focus here is on public reporting, 
it is assumed that these characteristics apply as well to reporting that occurs internal to a state 
or district, and certainly applies to required reporting of districts to states, and of states to the 
federal government. Five characteristics support Principle 4 (see Table 4). 
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Table 4. Principle 4 and Its Characteristics 



Principle 4. Public reporting includes the assessment results of all students. 

Characteristic 4.1. All students in all placement settings who receive educational services, 
regardless of severity of disability, are accounted for in the reporting system. 

Characteristic 4.2. The number and percentage of students with disabilities assessed and their 
aggregatable results are reported near to, as often as, and in ways similar to the reporting for 
students without disabilities. 

Characteristic 4.3. The number and percentage of students not assessed or whose results cannot 
be aggregated are revealed in public reports and explanations are given. 

Characteristic 4.4. Results from assessments administered in ways that raise policy questions are 
reported separately so that they can be publicly examined and discussed, as well as aggregated 
with other results. 

Characteristic 4.5. Reports are provided to educators, parents, students, policymakers, community 
members, and the media, with a clear explanation of results and implications. 



Characteristic 4.1. All students in all placement settings who receive educational services, 
regardless of severity of disability, are accounted for in the reporting system. 

Every student is counted. The basis for the counting of students is student enrollment. For stu- 
dents who are receiving special education services, the child count at a time closest to the time 
the assessment is administered typically is the basis for the count of all students. “All students” 
includes not only students in traditional public school placements, but also students who change 
schools or placements. All students who receive federally funded educational services in non- 
traditional settings are included and reported as well. These students include those in home 
schools, private schools, charter schools, state-operated programs, and in the juvenile justice 
system. The challenge of counting every student regardless of the severity of disability, and 
ensuring that each student’s progress counts, is fundamental to the success of standards-based 
reform. There is a national consensus that all students are to be held to high standards and all 
schools are to fully support all students’ efforts to reach those standards, regardless of the set- 
ting. If some students are excluded or set aside in reporting, the public has no way of knowing 
how all students or all schools are doing. 

This characteristic also means that every student counts, even if the student received an assess- 
ment result that could not be aggregated, was exempted by a parent, or did not count as a par- 
ticipant because of the use of a modification during testing. Despite the unfavorable outcomes 
that resulted from these conditions, the student still is part of the population and counts in the 
denominator when percentages of students assessed are calculated. 
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Just as the IEP enrollment for the school is the denominator when participation rates and profi- 
ciency rates of students with disabilities are calculated at the school level, and the IEP enrollment 
for the district is the denominator when participation rates and proficiency rates of students with 
disabilities are calculated at the district level, so too does the state IEP enrollment become the 
denominator when state participation rates and proficiency rates of students with disabilities 
are calculated for the state. Without a stable and consistent denominator, the results of some 
students are lost, and the participation and performance results become confusing at best and 
incomprehensible or misleading at worst. 



Characteristic 4.2. The number and percentage of students with disabilities assessed and their 
aggregatable results are reported near to, as often as, and in ways similar to the reporting for 
students without disabilities. 

The reporting of the number and percentage of students assessed and not assessed, and the re- 
porting of data on performance, by type of assessment, are provided as often as those data are 
reported for students without disabilities. This process includes specific reports of how many 
ELLs with disabilities participated (and did not participate) and their performance. All of these 
pieces of information are arranged in ways that are similar to those used for students without 
disabilities, and are provided as often as they are for students without disabilities. The goal is 
to ensure that public reporting is transparent and accessible for students with disabilities, just 
as much as it is for other students. Those reading public reports will better understand in this 
way that students with disabilities are not a group whose information and results are being 
hidden, but rather the system is checking on how these students are faring so that they are not 
overlooked as they were in the past. 



Characteristic 4.3. The number and percentage of students not assessed or whose results can- 
not be aggregated are revealed in public reports and explanations are given. 

At a minimum, every student who is not actually assessed in the assessment system is detect- 
able when results are reported. Typically, this identification is done by reporting the number of 
students not participating in the assessment system. Even if a state or district factors students 
who do not take the assessment into the reported results (e.g., by giving them a zero), the number 
of students excluded from participation is still reported. In addition, the reasons for exclusion 
(e.g., parent request, absenteeism, ELL exemption, noncompliance, cheating, procedural errors 
such as nonscorable test protocols due to administration or test company errors) are reported 
for students with disabilities. This characteristic does not preclude appropriate respect for con- 
fidentiality of individuals. For example, if reporting information on reasons for exclusion at 
the school level violates confidentiality, then the information is reported at the district level. If 
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confidentiality is violated because a state is reporting information by disability category, then 
the information on reasons for exclusion is reported only at the subgroup level (rather than by 
disability category). Regardless of where the confidentiality issue arises — if one does — there 
are clear indications of where the information on students not assessed, or whose results cannot 
be aggregated, are revealed in public reports. Further, explanations are given of the reasons for 
why results can not be reported. 



Characteristic 4.4. Results from assessments administered in ways that raise policy questions 
are reported separately so that they can be publicly examined and discussed, as well as ag- 
gregated with other results. 

When there are questions about a policy, such as when an accommodation is allowed even though 
its effects on the validity of results have not been determined, the results of the use of the accom- 
modation are as transparent as possible. Policy decisions about accommodations or other admin- 
istration considerations often must be made when the research literature is mixed in its evidence. 
Thus, policy decisions are made even though policy questions may remain. Accommodations 
frequently give rise to these policy questions. It may be determined, for example, that the use 
of a scribe is appropriate, but that there are questions about the use of the accommodation and 
whether it really is appropriate to aggregate results when the accommodation is used. There may 
be a belief that it should be used by a limited number of students, and a concern that if allowed 
the number of students using the accommodation will increase dramatically to the point that it 
is being used by students for whom the accommodation is inappropriate. In this situation, even 
though the policy decision has been to allow the accommodation and aggregate the results — the 
results are also disaggregated so that they can be publicly examined and discussed. 



Characteristic 4.5. Reports are provided to educators, parents, students, policymakers, com- 
munity members, and the media, with a clear explanation of results and implications. 

State and district staff members have a responsibility to ensure that data are used in ways that 
are consistent with the purpose of each assessment. Reports are readily available and acces- 
sible, and include cautions about misinterpretation of data. Particular care is taken to ensure 
that reports are available and accessible to linguistically and culturally diverse parents. This 
task entails making information available in hard copy and a variety of formats and languages. 
States and districts provide assistance interpreting the results. If tests are designed to yield the 
most accurate data at the classroom or school level, all student level reports will specify the 
necessity of using data from multiple sources (e.g., from classroom assessments or specific 
diagnostic tools) for individual students. 
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Consideration is given to having community information sessions or special outreach to the media 
to help people use the reports responsibly. This process may be especially important when there 
are new approaches to data, such as growth models for accountability, where it may be more 
difficult to include students with disabilities because of high mobility. Clear reporting of these 
issues, including when students are lost to inclusion in the data reports because of mobility, and 
the characteristics of those students who are dropped, is part of public reporting. 

Finally, for students in placements other than the local school, students are included in reports 
that will most directly affect the student’s education — where his or her performance counts, 
and where public reporting can make a difference. For example, if a student with disabilities is 
being served in a specialized setting outside of his or her home district (or school), the progress 
of that student is reported in the context where accountability and concern for that student most 
directly lies, in other words, in the student’s home school (the school that the student would 
have attended if he or she did not have a disability). 



Principle 5. Accountability determinations are affected in the same way by all 
students. 

This principle provides the second level of accountability for students with disabilities, that of 
school, district, and state accountability. State accountability plans that promote equal access 
and opportunity for all students and increased expectations for schools ensure that assessment 
participation and performance data are integrated into district and state accountability determina- 
tions in the same way for all students. Three characteristics support Principle 5 (see Table 5). 

Table 4. Principle 4 and Its Characteristics 



Principle 5. Accountability determinations are affected in the same way by all students. 

Characteristic 5.1. Performance data for all students factor into accountability determinations 
regardless of how they were assessed or why they were not assessed. 

Characteristic 5.2. Accountability plans treat all groups of students the same. 

Characteristic 5.3. Formal and informal accountability reports focus on identifying areas 
and activities that can be changed to improve student learning rather than implying that low 
performance is attributable to student characteristics. 
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Characteristic 5.1. Performance data for all students factor into accountability determinations 
regardless of how they were assessed or why they were not assessed. 



All assessment results that can be validly aggregated contribute in a similar way to account- 
ability determinations regardless of how students were assessed. Using assessment practices 
such as accommodations or alternate assessments that the state has determined to be valid for 
this use does not diminish the impact students’ results have on accountability determinations. 
Students who have assessment results that cannot be aggregated or who have no assessment 
results are also included in accountability determinations. The reasons students lack results that 
can be aggregated are reported and examined in ways that promote increasing the percentage 
of results that can be aggregated. 



Characteristic 5.2. Accountability plans treat all groups of students the same. 

Accountability plans are based on the same assumptions for all groups and eliminate any 
implementation guidelines that systematically exclude or reduce the impact of some groups. 
Practices that run the risk of doing this by making low group performance invisible or accept- 
able are rejected, such as adjusting target performance for group characteristics or looking only 
at changes in test performance. Low group performance remains visible by avoiding arbitrarily 
large minimum group size reporting requirements or by using average scores or various kinds 
of indices. Decisions about procedures proposed to protect the privacy of the students and 
to produce sound accountability decisions are subjected to independent review for technical 
adequacy to ensure that states, districts, and schools are transparent in their performance and 
provide for all students to affect accountability determinations equitably. 



Characteristic 5.3. Formal and informal accountability reports focus on identifying areas 
and processes that can be changed to improve student learning rather than implying that low 
performance is attributable to student characteristics. 

Accountability reports show results broken out by student groups, grade levels, content areas, 
districts, and schools. District and school information shows how student performance relates 
to the opportunities students have to learn the challenging grade-level content, and the training, 
resources, state improvement plans and activities, and other supports available for these schools, 
teachers, and students. States support, train, and expect educators at all levels to respond to ac- 
countability reports by accelerating and scaffolding student learning to improve access of every 
learner to the grade- level content. Reports that merely identify groups of low performing students 
and that are presented without support for effective use for school improvement could lead to 
blaming and excuse-making that hinders progress toward the goal of success for all students. 
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Principle 6. Continuous improvement, monitoring, and training ensure the quality of 
the overall system. 

The value of the assessment system is documented and strengthened over time through continu- 
ous monitoring, training, and adjustments in all aspects of the assessment and accountability 
system. This principle addresses the need to base inclusive assessment practices on current and 
emerging research and best practice, with continuous improvement of practices as research- 
based understanding evolves. Because society is expecting more of traditional large-scale 
assessments and requiring multiple uses of test results, we must invest time and thought into 
improving them. It requires addressing potential threats to validity from the design of the as- 
sessment, development of participation guidelines and training, administration procedures, and 
monitoring of implementation practices. 

By working together on improvement of inclusive large-scale assessments for system account- 
ability, stakeholders can sustain commitment to keeping the standards high and keeping the 
focus clear on all students being successful. Ongoing training of IEP team members and other 
key partners is an essential component of this effort. Four characteristics support Principle 6 
(see Table 6). 

Table 6. Principle 6 and Its Characteristics 



Principle 6 Continuous improvement, monitoring, and training ensure the quality of the 
overall system. 

Characteristic 6.1. The quality, implementation, and consequences of student participation 
decisions are monitored and analyzed, and the data are used to evaluate and improve the quality 
of the assessment process at the school, district, and state levels. 

Characteristic 6.2. States provide training to multiple stakeholders about the implications of use 
of available assessment options to improve IEP team decision making about how the student 
participates in the large scale assessment for accountability. 

Characteristic 6.3. The use that is made of accountability reports and the impact that 
accountability decisions have on educational processes and student learning are monitored to 
determine what adjustments may be needed to improve the accountability system. 

Characteristic 6.4. The quality of assessment tools is continuously evaluated and improved by 
applying information gathered about the use and impact of assessment results and by responding 
to developments in the field of measurement. 
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Characteristic 6.1. The quality, implementation, and consequences of student participation 
decisions are monitored and analyzed, and the data are used to evaluate and improve the qual- 
ity of the assessment process at the school, district, and state levels. 

Identifying methods to use at the school level to check on decision-making patterns, and provid- 
ing feedback to IEP teams on appropriateness of decisions, improves the quality of assessment 
data in the long term. Likewise, if good participation, accommodation, and alternate assess- 
ment decisions are made at the IEP team level, but the information is poorly documented, not 
communicated to instructional settings or to assessment personnel, the validity of the assess- 
ment results may be affected. By monitoring these decisions, and ensuring the decisions are 
implemented appropriately, schools, districts, and states ensure the best possible measurement 
of actual student progress toward standards. 

Across the state, test administration procedures and forms capture essential data for determin- 
ing the characteristics of students, accommodations used by the student for some or all parts of 
the test, or ways the student was included in alternate assessments. Capturing data of student 
characteristics and use of accommodations for all or parts of the test yield essential data in deter- 
mining the validity of the test for these students specifically, and contribute to the research base 
on effects of accommodations on the validity of the results more generally. Understanding the 
characteristics of students who participate in alternate assessment options assists in validation 
of the assessment approach for the participants, as a group and as individuals, and in identifica- 
tion of a need for adjustment of the approach. It also provides a statewide profile of patterns 
of decision making and use of participation options and leads to systematic intervention with 
schools where unusual patterns of participation are occurring. 

In developing systems, the view of consequences often depends on the perspective of the viewer. 
For that reason, the ongoing monitoring and evaluation of consequences requires stakeholder 
involvement to determine which consequences are intended or unintended, and which are positive 
or negative. A systematic process for consequential validity studies is built into state procedures, 
which builds support for changes in the systems as they are needed. 



Characteristic 6.2. States provide training to multiple stakeholders about the implications of 
use of available assessment options to improve IEP team decision making about how the student 
participates in the large scale assessment for accountability. 

All IEP teams and other key personnel have access to ongoing training and technical assistance. 
State departments of education make connections, provide leadership and incentives, develop 
written materials, and present introductory workshops, but day-to-day support is built into a 
district’s comprehensive system of professional development. In addition, states partner with 
institutions of higher learning to rethink basic teacher competency and licensure requirements 
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in light of the new emphasis on measuring the progress of all students toward high standards. 
Parent training organizations and other advocacy groups are essential training partners to reach 
parents and the students themselves. 

Increasing the assessment literacy of IEP team members improves the quality of the assessment 
decisions made by each team. Increased assessment literacy, in turn, improves how well assess- 
ments measure progress toward standards for all students, regardless of how they participate 
(with or without accommodations, or in an alternate assessment). Ultimately, the validity of the 
assessment results for use in system accountability rests on these individual student participa- 
tion decisions. 



Characteristic 6.3. The use that is made of accountability reports and the impact that account- 
ability decisions have on educational processes and the learning of students with disabilities 
are monitored to determine what adjustments may be needed to improve the accountability 
system. 

Information is gathered from districts and schools indicating how reports have been used and 
what actions have been taken in response to reports. Such information is reviewed when new test 
results are obtained and it is related to the performance of students with disabilities. Evaluations 
of educators’ responses to the accountability reports and decisions and their impact on student 
learning are used to determine what additional staff development or supports or other changes 
in the accountability system may be needed to continue improving student learning. 



Characteristic 6.4. The quality of assessment tools is continuously evaluated and improved by 
applying information gathered about the use and impact of assessment results and by respond- 
ing to developments in the field of measurement. 

States monitor how schools implement assessments and how they use and respond to assessment 
results to see where assessment practices and tools need to be improved. States also remain 
informed about federal requirements, guidance, and options. States seek solutions to improving 
assessment tools and practices by working with other states and with experts in the fields of 
assessment, curriculum and instruction, and special populations. 
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Core Resources 



Note: The National Center on Educational Outcomes has been documenting the participation 
and performance of students with disabilities in large-scale assessments for almost two decades. 
Over that time span, we have published results of data analyses, policy analyses, and systemati- 
cally documented changing practices of inclusive assessment. The Principles reflect the extensive 
bibliographies of our publications over the time period. The list below was chosen to reflect key 
references that have been commonly included in our publications, focusing on seminal works 
on large-scale assessment, public policy on standards-based reform, testing, and students with 
disabilities, literature reviews, and specific NCEO publications that summarize key issues that 
are addressed in the Principles. This list is not exhaustive, but reflects a core body of work that 
has influenced our thinking. We will continue to update core references in our Web version of 
the Principles as we develop and publish companion resources for these Principles. 
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DC: American Educational Research Association. 

Albus, D. A., & Thurlow, M. L. (2007). English language learners with disabilities in state 
English language proficiency assessments: A review of state accommodation policies (Synthe- 
sis Report 66). Minneapolis, MN: University of Minnesota, National Center on Educational 
Outcomes. 

Bolt, S. E., & Roach, A. T. (2008). Inclusive assessment and accountability: A guide to ac- 
commodations for students with diverse needs (The Guilford Practical Intervention in Schools 
Series). New York, NY: Guilford Publications, Inc. 

Bolt, S. E., & Thurlow, M. L. (2004). Five of the most frequently allowed testing accommoda- 
tions in state policy. Remedial and Special Education 25 (3), 141-152. 

Browder, D. M., Wakeman, S. Y., Flowers, C., Rickelman, R., Pugalee. D., & Karvonen, M. 
(2007). Creating access to the general curriculum with links to grade level content for students 
with significant cognitive disabilities: An explication of the concept. Journal of Special Educa- 
tion, 47(1), 2-16. 

Christensen, L. L., Lazarus, S. S., Crone, M., & Thurlow, M. L. (2008). 2007 state policies on 
assessment participation and accommodations for students with disabilities (Synthesis Report 
69). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. 

DeStefano, L., Shriner, J. G., & Lloyd C. (2001). Teacher decision making in participation of 
students with disabilities in large-scale assessment. Exceptional Children, 65(1), 7-22. 
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Fast, E. F., Blank, R. K., Potts, A., & Williams, A. (2002). A guide to effective accountability 
reporting. Washington, DC: Council of Chief State School Officers. 

Haertel, E. H. (1999). Validity arguments for high stakes testing: In search of the evidence. 
Educational Measurement: Issues and Practices, 18(4), 5-9. 

Heubert, J. R, & Hauser, R. M. (1999). High stakes: Testing for tracking, promotion, and gradu- 
ation. Washington, DC: National Academy Press. 

Johnstone, C. J., Altman, J., Thurlow, M. L., & Thompson, S. J. (2006). A summary of research 
on the effects of test accommodations: 2002 through 2004 (Technical Report 45). Minneapolis, 
MN: University of Minnesota, National Center on Educational Outcomes. 

Kane, M. (2002). Validating high-stakes testing programs. Educational Measurement: Issues 
and Practices, 21(1), 31-41. 

Kane, M. (2006). Validation. In R. L. Brennan (Ed). Educational measurement (4 th edition). 
Washington, DC: American Council on Education/Praeger. 

Kleinert, H., Browder, D., & Towles-Reeves, E. (2005). The assessment triangle and students 
with significant cognitive disabilities: Models of student cognition. Lexington, KY: University 
of Kentucky, National Alternate Assessment Center. 

Lane, S., Park, C. S., & Stone, C. (1998). A framework for evaluating the consequences of as- 
sessment programs. Educational Measurement: Issues and Practices, 17(2), 24-28. 

Lazarus, S.S., Thurlow, M.L., Lail, K.E., & Christensen, L. (2008). A longitudinal analysis of 
state accommodations policies: Twelve years of change 1993-2005. Journal of Special Edu- 
cation. Sage Journal Online First published May 12 at: http://sed.sagepub.com/cgi/rapidpdf/ 
00224669073 13524vl; Hard copy: forthcoming, will be in 43(2). 

Marion, S., & Pellegrino, J. (2006). A validity framework for evaluating the technical quality 
of alternate assessments. Educational Measurement: Issues and Practice, 25(4), 47-57. 

McDonnell, L. M., McLaughlin, M. J., & Morison, P. (1997). Educating one and all: Students 
with disabilities and standards-based reform. National Academy Press, Washington D. C. 
(p. 7). Retrieved February 21, 2004, from http://www.netlibrary.com. 

McGrew, K. S., & Evans, J. (2004). Expectations for students with cognitive disabilities: Is the 
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University of Minnesota, National Center on Educational Outcomes. 
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